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latter being a college preparatory diploma. Starting in 2017, the Virginia Department of Education (VDOE) 
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credential to encourage them to pursue opportunities that enhance their career readiness. This is likely to be 
particularly important for students graduating with the Standard diploma, as they have been shown to have 
limited success in postsecondary education. 


This study examined the CTE credentials Virginia high school graduates most commonly earned from 2011 
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period, but the percentage of students earning the Workplace Readiness Skills (WRS) and W!se Financial 


Literacy Certification credentials increased. Both of these credentials cover broad skills relevant to a wide range 
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Standard diploma graduates, there were few differences in the top 10 credentials by diploma type, both in 
terms of which credentials were most common as well as the rates at which students earned these credentials. 
Regardless of diploma type, in 2017, 9 of the top 10 credentials were broad credentials that were not narrowly 
aligned to a specific occupation or industry. This study also looked at the top 10 credentials earned by 2017 
Standard diploma graduates across a variety of student subgroups, including English learner students, 
economically disadvantaged students, students with disabilities, and racial/ethnic subgroups. English learner 
students and students with disabilities earned the top 10 credentials at lower rates than other Standard 
diploma graduates. Student credential-earning rates differed the most by geographic region, both in terms of 
which credentials appeared in the top 10 and the percentage of students earning the top 10 credentials. 


This study highlights the need for additional analyses to help CTE stakeholders and policymakers understand 
the value of different types of CTE credentials. In particular, Virginia and other states might explore the relative 
value of broad CTE credentials that apply to a wide range of jobs and have become increasingly prevalent in 
Virginia compared with CTE credentials that are more narrowly aligned with a specific occupation or industry. 


Virginia is one of several states leveraging readily available assessments that include industry credentialing, state 
licensure, occupational competency, and workplace-readiness assessments for high-school-age students as a way 
to monitor their preparation for college and careers. For more than a decade, the Virginia Department of 
Education’s (VDOE’s) career and technical education (CTE) programs have provided students with opportunities 
to earn CTE credentials, including credentialing assessments and state licensures in certain areas. The state 
invested in a long-term effort to increase the use of these assessments in high school and eventually added earning 
a CTE credential as a requirement to graduate from high school with a Standard diploma (see box 1 for definitions 
of key terms). The Standard diploma is one of Virginia’s two main diploma types,‘ and 41 percent of all high school 
graduates earned a Standard diploma in 2019 (VDOE, n.d.a). Like other states, Virginia’s CTE credential options 


1 Additional types of diplomas are available to certain students, such as students with disabilities. 


offer students a range of ways to demonstrate skills and competencies, all of which require students to pass a test 
to verify their learning. Some, but not all, of the state’s approved measures lead to industry-recognized 
certification and professional licenses. The state also offers students opportunities to demonstrate skills learned 
through competency measures and general measures of workplace and college preparatory skills. 


Box 1. Key terms 


Advanced Studies diploma. The Advanced Studies diploma is one of two diploma types available to all Virginia high school 
students and is considered a rigorous college preparatory high school diploma (Holian & Mokher, 2011; Jonas et al., 2012; 
Jonas, Garland, & Yamaguchi, 2014). Students must earn at least 26 credits, including four credits each of English, 
mathematics, science, and history, and three credits of a world language (Virginia Department of Education [VDOE], 2020a). 
Advanced Studies diploma earners must also pass associated end-of-course Standards of Learning tests or other 
assessments approved by the state Board of Education for five of these courses (VDOE, 2020a). 


Career and technical education (CTE). Virginia defines CTE as programs “designed to prepare young people for productive 
futures while meeting the commonwealth's need for well-trained and industry-certified technical workers” (VDOE, 2020b). 
Virginia includes CTE courses within 16 career clusters, each with multiple pathways designed by the school divisions. For 
example, the Health Science career cluster may include pathways for therapeutic services, biotechnology, or diagnostic 
services. 


CTE credential. A CTE credential certifies that a student has mastered specific CTE content. In Virginia, students can earn a 
CTE credential by passing a qualifying assessment. VDOE uses a broad definition of CTE credentials, which includes industry 
credentials, state licensure examinations, national occupational competency assessments, or the Virginia workplace- 
readiness skills assessment (VDOE, 2020b). Virginia law requires the Board of Education to maintain a list of approved 
credentials students can earn to meet this graduation requirement. Not all of Virginia’s CTE credentials result in industry and 
professional certifications, although the Code of Virginia requires CTE programs to be aligned with these certifications where 
they exist (Code of Virginia, 2006). (Refer to appendix A for more information on CTE credentials in Virginia.) 


CTE credential earner. A CTE credential earner is a student who has completed the requirements to earn a CTE credential by 
taking and passing an approved CTE credential assessment. 


CTE credential type. VDOE divides credentials into four types, each of which is awarded after students pass a Board of 
Education—approved assessment, as shown in table 1. 


Economically disadvantaged. An economically disadvantaged student is “any student who: (1) is eligible for free or reduced- 
price meals, or (2) receives Temporary Assistance for Needy Families (TANF), or (3) is eligible for Medicaid, or (4) [is] identified 
as either migrant or experiencing homelessness at any point during the school year” (Virginia Longitudinal Data System, 
2019). 


Federal program participation. For the purposes of this study, federal program participation refers to participation in federal 
programs for English learner students, economically disadvantaged students, and students with disabilities, and includes 
students who were eligible for these programs at any point during their enrollment in a Virginia high school. 


Narrowly aligned credentials. The study team considered credentials to be narrowly aligned if they support preparation for 
a specific occupation or industry, as opposed to broad credentials that can apply a wide range of occupations. 


Region. VDOE created eight geographic regions 
(“superintendent regions”), each represented by a 
regional superintendent. These regional 
superintendents provide input to the state 
superintendent and executive team on key issues of 
policy and practice, and communicate policies 
enacted by the state superintendent to their local 
school divisions. 


Regions in Virginia 


Standard diploma. The Standard diploma is one of E\ 3h < Ws = 
two diploma types available to all Virginia high ; | : 7s : 
school students. To graduate with a Standard = 2] AeA 
diploma, students who entered grade 9 for the first J Source: Virginia Department of Education (n.d.b) 

time in 2011/12 were required to earn at least 22 
credits, including four credits in English, three credits each in mathematics, laboratory science, and history and social sciences, 
and two credits in a world language. Standard diploma earners must also successfully pass associated end-of-course 
Standards of Learning tests or other assessments approved by the state Board of Education for five of these courses (VDOE, 
2020b). Students who graduated on time in 2017 with a Standard diploma were the first students required to earn a CTE 
credential (VDOE, 2020b). 


Top credentials. The most commonly earned credentials identified in the analysis. Table 1 describes the top credentials that 
appear in the findings. 


VDOE anticipated that adding the CTE credential as a graduation requirement would potentially increase students’ 
attainment of workplace and technical skills (Virginia Department of Education, 2016); VDOE also suggested that 
earning this credential might spur graduates to achieve more advanced certification requiring additional 
postsecondary education and training (Virginia Department of Education, 2008). CTE credentials might also 
strengthen graduates’ employment outcomes if the credentials validated skills aligned with labor-market needs. 
Unfortunately, these conditions are not always met. Two recent national studies found that many high school 
students’ CTE concentrations and credentials were not well aligned to the labor market (ExcelinED & Burning Glass 
Technologies, 2019; Sublett & Griffith, 2019). Using data from 24 states, one study estimated that only 19 percent 
of credentials high school graduates earned were in demand, and noted that general career-readiness credentials 
had little to no value in the labor market based on clearly identifiable qualifications listed in employers’ job 
postings (ExcelinED & Burning Glass Technologies, 2019). Licenses, on the other hand, are most likely to be valued 
by employers because they are either required for the occupation or signal to the employer that an individual has 
skills that are highly valued in the industry (ExcelinED & Burning Glass Technologies, 2019). 


VDOE created four categories of credentials and approved 471 credentials for the 2016/17 school year (VDOE, 
2016). The categories VDOE uses do not necessarily align with credential definitions that other locales use. VDOE’s 
four categories are: 


e Industry credential. Industry credentials include industry certifications, which may be from a recognized 
industry, trade, or professional association, and pathway industry certifications. Pathway industry 
certifications include a series of credentials from a recognized industry that, when completed, lead to a 
full industry certification (VDOE, 2018). VDOE’s definition of industry credentials includes credentials that 
apply to a broad range of industries, including general career readiness credentials, and industry-specific 
credentials. This definition does not conform to the way many other states define an industry credential 
(ExcelinED & Burning Glass Technologies, 2019). 


e State licensure. For the purposes of this study, state licensure is a CTE credential that also confers a state- 
recognized license, such as a license to practice as a cosmetologist. 


Workplace Readiness Skills (WRS) assessment. The WRS measures general career readiness skills and 
satisfies Virginia’s requirement for a CTE credential for graduation with a Standard diploma. It assesses 
three domains: Personal Qualities and Abilities, Interpersonal Skills, and Professional Competencies 
(Virginia’s CTE Resource Center, 2020). 


National Occupational Competency Testing Institute (NOCTI) assessment. Virginia students can meet the 
CTE credential requirement by completing NOCTI “Job Ready” assessments, such as Accounting—Basic and 
Accounting—Advanced, Collision Repair and Refinishing Technology Assessment, and Computer Aided 
Design. According to NOCTI (2020), its Job Ready assessments assess technical skills at the occupation 
level, measure aspects of occupational competence such as factual and theoretical knowledge, and as a 
group, aim to assess the skills at the secondary and postsecondary level. Students can also meet the CTE 
credential by passing NOCTI Pathway assessments, such as the Banking and Related Services Assessment. 
Pathway assessments are broader in scope than the Job Ready assessments and aim to measure the 
technical skills within a pathway or cluster as well as soft skills and academic skills contextualized to the 
occupation. 


Some credentials broadly measure a variety of content areas (for example, the National Career Readiness 
Certificate and the WRS assessment), while others are more narrowly aligned with a particular occupation or 
industry (such as the cosmetology licensure). 


To provide context for this study, table 1 describes the credentials that appear in the findings, grouped by the 
credential type, using categories assigned by VDOE. The table also shows industry alignment for each credential, 
identifying whether the credential is narrowly aligned to a specific occupation or industry (narrowly aligned 
credentials) or instead applies to a wide range of occupations (broad credentials). 


Table 1. Description of top credentials identified in the study, including whether the credential is narrowly or 
broadly aligned to a specific occupation or industry, categorized by Virginia Department of Education (VDOE)- 
assigned credential type 
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occupation 
/industry 


Credential name alignment? Test or measure used to earn credential 
VDOE credential type: Industry credentials 


American Association of Family Broad Test that assesses a broad range of content areas related to careers 

Consumer Sciences (AAFCS): Broad Field involving human services; consumer services, protection, and advising; 

Family and Consumer Services education and training; and social and community services. 

Armed Services Vocational Aptitude Narrow An aptitude test that compiles career information from a variety of 

Battery sources to help students make postsecondary career plans. The military 
requires recruits to take this exam. 

Automotive Service Excellence (ASE) Narrow Entry-level certification exam testing the skills necessary to diagnose, 

Student Certification Automotive: Brakes service, and repair brakes in cars, sport utility vehicles, and light-duty 
trucks. 

ASE Student Certification Automotive: Narrow Entry-level certification exam testing the skills necessary to successfully 

Maintenance and Light Repair perform the most common maintenance and light repair tasks. 

ASE Student Certification Automotive: Narrow Entry-level certification exam testing the skills necessary to diagnose, 

Suspension and Steering service, and repair suspension and steering in cars, SUVs, and light-duty 
trucks. 

Beef Quality Assurance Certification Narrow Credential that warrants the holder is knowledgeable about best 


management practices to improve the safety and quality of beef and 
allows him or her to become a beef quality assurance certified producer. 


College and Work Readiness Assessment = Broad Exam that uses performance tasks to measure critical thinking and 


written communication skills, including problem solving, scientific and 
quantitative reasoning, critical reading and evaluation, critiquing an 
argument, and writing mechanics and effectiveness. 


Credential name 


NEVaaehwae) g 
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occupation 
/industry 
alignment? 


Test or measure used to earn credential 


Customer Service and Sales Certification 


Microsoft Office Specialist (MOS) Excel 


MOS PowerPoint 


MOS Word 


National Career Readiness Certificate 


National Construction Career Test Core: 
Introductory Craft Skills 
ServSafe Manager Certification 


SkillsUSA: Customer Service Examination 
W!se Financial Literacy Certification 


VDOE credential type: National 
Occupational Competency Testing 
Institute 

Cosmetology Assessment 


VDOE credential type: State licensure 
Cosmetology Licensure 
National Nurse Aide Assessment Program 


Private Applicator Certification 


VDOE credential type: Workplace 
Readiness Skills (WRS) 
WRS Assessment 


Narrow 


Broad 


Broad 


Broad 


Broad 


Broad 


Narrow 


Broad 


Broad 


Narrow 


Narrow 


Narrow 


Narrow 


Broad 


Students must complete an online course and pass an exam to earn this 
certificate. The course and exam cover topics such as customer life cycle, 
developing strategies to engage customers, assessing customer needs, 
and closing sales. 

Assesses ability to use Excel to create and manage worksheets and 
workbooks; manage data cells and ranges; create tables; perform 
operations with formulas and functions; and create charts and objects. 
Assesses ability to use PowerPoint to create and manage presentations; 
insert and format text, shapes, and images; insert tables, charts, 
SmartArt, and media; apply transitions and animations; and manage 
multiple presentations. 

Assesses ability to use Word to create and manage documents; format 
text, paragraphs, and sections; create tables and lists; create and 
manage references; and insert and format graphic elements. 
Certificate that requires completion of assessments in applied math 
(measures critical thinking, mathematical reasoning, and problem 
solving), graphic literacy (measures skills needed to locate, synthesize, 
and use information from graphics), and workplace documents 
(measures skills needed to read and use workplace documents such as 
memos and letters). 

Assesses introductory craft skills including: safety, math, tools, 
construction drawings, rigging. 

Assesses a variety of tasks needed by restaurant managers, including 
management of food safety practices; hygiene and health; safe receipt, 
storage, transportation, and disposal of food; safe preparation and 
cooking of food; safe service and display of food; cleanliness and 
sanitation; and facilities and equipment. 

Assesses a variety of workplace skills in four main domains: 
decisionmaking, teamwork, communication, and leadership. 

Exam accompanying a course on personal finance and designed to 
demonstrate that students are financially savvy. 


Exam that measures job-ready skills, including factual and theoretical 
knowledge. This assessment measures six areas of cosmetology: safety 
and sanitation, scientific concepts, salon-related business skills, physical 
services, chemical services, hair designing. 


Exam required to become a cosmetologist in Virginia. 


This licensure exam is required to become a certified nursing assistant in 
Virginia. 

Certifies holders as private pesticide applicators and is required to apply 
restricted-use pesticides, to produce an agricultural commodity, or to 
apply pesticides on their own land or that of an employer. 


Measures a variety of work-related skills across three main domains: 
Personal Qualities and Abilities, Interpersonal Skills, and Professional 
Competencies. 


*The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. Note: 


Table includes a description of any credential that appears in the findings. This study uses the credential type as defined by the Virginia Department of 


Education. Refer to Virginia’s CTE [Career and Technical Education] Resource Center (2020) for more information these credentials, including which 


organizations sponsored them. 


Source: Authors’ creation using information from Virginia’s CTE Resource Center (2020). 


Earning credentials that can increase job opportunities after high school is likely to be more beneficial for Virginia’s 
students who graduate with a Standard diploma than for those earning a college preparatory diploma (the 
Advanced Studies diploma). Compared to students earning the Advanced Studies diploma, Standard diploma 
earners are less likely to enroll in, persist, or complete postsecondary education programs, including one-, two-, 
and four-year programs (Garland, LaTurner, Herrera, Jonas, & Dougherty, 2011; Holian & Mokher, 2011; Jonas & 
Garland, 2014; Jonas, Garland, & Yamaguchi, 2014).2 One study showed that four years after high school 
graduation, 46 percent of Advanced Studies diploma graduates earned a postsecondary credential? whereas only 
8 percent of Standard diploma graduates earned a postsecondary credential (Jonas & Garland, 2014). 


Recognizing the limited success of Standard diploma earners in postsecondary education and training programs, 
it is critical to examine the CTE credentials these graduates earn in high school as a first step toward understanding 
how well these credentials provide them with the foundation needed for long-term, meaningful employment. In 
addition, given that the state has been working to increase the use of CTE assessments in high school for more 
than a decade (see exhibit 1), Virginia stakeholders are interested in examining CTE-credential-earning patterns 
over time. This information can inform ongoing legislative efforts to strengthen CTE credential policy and inform 
state and local support for implementing policy to maximize benefits to students. For example, stakeholders could 
examine patterns in CTE credentials earned relative to evolving employment opportunities to see if earned 
credentials align with state or regional employer needs. 


Students who earn the Standard diploma are disproportionately economically disadvantaged, English learners, 
and students with disabilities, and are more likely to be Black or Hispanic than from other racial/ethnic groups. In 
2019, for example, fewer than half of all high school graduates (41 percent) earned Virginia’s Standard diploma, 
yet 60 percent or more of graduates who were English learner students, economically disadvantaged students, or 
students with disabilities earned the Standard diploma (table 2). Further, among Virginia’s racial/ethnic minority 
groups, more than half of Virginia’s Black and Hispanic graduates earned Standard diplomas, compared to 38 
percent or less of students who are Asian, White, or two or more races. Given these differences, it is important 
for studies examining CTE credentials in Virginia to disaggregate data for these groups, which are 
disproportionately subject to the policy and potentially have the most to gain. 


Table 2. Percentage of Virginia’s 2019 cohort graduates earning Advanced Studies and Standard diplomas 


Graduates earning 


Advanced Studies Graduates earning Percentage point 
Student group (elf e)Ke) age mm ( exsiKexs1a19) SyElateFeIxeWel] edColaarein( Xeicexsa19) difference 
All students 56.3 40.7 15.6 
English learner students 35.1 59.6 -24.5 
Economically disadvantaged 10.8 63.6 -52.8 
Students with disabilities 21.4 71.7 -50.3 
Asian 78.9 19.6 59.3 
Black 39.6 55.2 -15.6 
Hispanic 43.9 53.1 -9.2 
White 63.0 34.6 28.4 
Two or more races 59.0 38.4 20.6 


Note: Some Virginia high school graduates are eligible to earn other types of diplomas, which is why none of the rows add up to 100 percent (Virginia 
Department of Education, 2020b). 
Source: Authors’ calculation based on Virginia Department of Education State Cohort Reports (n.d.a). 


? See box 1 for a description of the two main diploma types, Standard and Advanced Studies diplomas. 
3 Jonas and Garland (2014) included certificates, associate’s, and bachelor’s degrees from a postsecondary institution in 
their definition of postsecondary credential. 


Research questions 

To help Virginia stakeholders examine patterns in the types of CTE credentials students earned over time as well 
as differences by diploma type and among student groups, the study focused on the most commonly earned 
credentials (see box 2 for a description of the study’s data sources, sample, and methods). Specifically, this study 
addressed the following research questions: 


1. For the 2011-2017 graduates, what were the top five CTE credentials earned and how did they change over 
time? 

2. For the 2017 graduates, what were the top 10 most commonly earned CTE credentials earned overall and by 
diploma type? 

3. For 2017 Standard diploma earners, how did the top 10 CTE credentials vary across different student 
characteristics (such as demographics and federal program participation) and by region? 

The current report is the first of a two-report series. This report focuses on the most commonly earned 

credentials. The second report will answer additional questions about credential-earning rates as a whole and 

examine their correlation with postsecondary enrollment. 


Box 2. Data sources, sample, and methods 


Data sources. The Virginia Department of Education provided the data for these analyses through the Virginia Longitudinal 
Data System. The de-identified administrative records were at the student level and included school enrollment data, student 
demographic variables, and career and technical education (CTE) credential assessment records. 


Sample. For research question 1, the study population consisted of all Virginia public high school graduates who received a 
diploma between 2011 and 2017. The rest of the analyses included only the 2017 high school graduates because this was the 
first year graduates were subject to the new Standard diploma requirement to earn a CTE credential. The analysis for research 
question 3 included only graduates who earned a Standard diploma in 2017 to focus on results directly related to the CTE 
credential policy. 


Methodology. The study used descriptive statistics to answer the research questions. For each question, the study team 
identified the top credentials by calculating the percentage of students who earned each type. For research question 1, for 
each year, the study team calculated the percentage of graduates who earned each credential and identified the five 
credentials with the highest percentages. For research question 2, the study team identified the 10 credentials that had the 
highest percentage of students earning them, overall and by diploma type (Standard and Advanced Studies). This part of the 
analysis includes the top 10 credentials to provide stakeholders with additional information about which ones students 
earned after a CTE credential became a requirement for the Standard diploma. For research question 3, the study team 
identified the 10 credentials earned by the highest percentage of students who earned a Standard diploma within various 
subgroups and within each region. The student groups included racial/ethnic groups and students identified at some point in 
a Virginia high school as English learner students, economically disadvantaged students, or students with disabilities. For 
more detailed information on the methodology, refer to appendix B. 


Findings 

This section presents the main findings for this study. Appendix C includes supporting analyses that address the 
research questions. Appendix D includes additional analyses not directly related to the study’s research questions 
but that Virginia stakeholders might find valuable. To provide context for the findings, refer to the description of 
the top credentials in table 2. 


Although the five most commonly earned CTE credentials in Virginia remained the same from 2011 to 
2017, the percentage of students earning the WRS and W!se Financial Literacy Certification 
credentials increased by more than 30-fold while the percentage of students earning the other 
credentials remained relatively stable. 


From 2011 to 2017, the top five credentials graduates earned were the W!se Financial Literacy Certification, WRS 
assessments, Customer Service and Sales Certification, Microsoft Office Specialist (MOS) Word, and MOS 
PowerPoint examinations (see figure 1).* 


The biggest change in credential-earning rates occurred between 2014 and 2015, before the new requirement 
went into effect. The percentage of students earning the WRS and W!se Financial Literacy Certification credentials 
primarily drove this change. The percentages of students earning the other top three credentials were relatively 
stable over time. During this same time period, the percentage of students who did not earn at least one CTE 
credential fell from 57 percent to 30 percent. 


The percentage of graduates earning the WRS credential started to increase in 2012, when it became the most 
frequently earned CTE credential for three years (see figure 1). WRS is an assessment option that Virginia helped 
develop to align with the statewide workplace-readiness-skills framework, which defines a set of personal 
qualities, people skills, and professional abilities Virginia employers and educators identified as essential for 
success in the workplace (Crespin, 2019). After decades of research to define the skills constituting workplace 
readiness, Virginia collaborated with curriculum and assessment developers to develop materials to teach and 
assess these skills. Virginia conducted a pilot of the WRS assessment in fall 2010, and the Virginia Board of 
Education adopted it for statewide use in 2011 (Career and Technical Education Consortium of States, 2011). 


In 2015, the W!se Financial Literacy Certification credential became the most frequently earned CTE credential in 
Virginia. The prevalence of the W!se credential is associated with a new graduation requirement: all students 
entering grade 9 during the 2011/12 school year or later had to pass the Economics and Personal Finance course 
to graduate, and the learning standards for this course align with the W!se Financial Literacy Certification. Some 
schools administer the W!se assessment in this course, which ensures students earn a board-approved CTE 
credential. 


“ Students can earn more than one credential, which is why the percentages in figure 1 can total to more than 100 percent 
for a given year. Of the students who earned a Standard diploma in 2017, 54 percent earned one credential, 26 percent 
earned two credentials, and 13 percent earned more than two credentials (figures D1 and D2 in appendix D). 


Figure 1. Percentage of 2011-2017 Virginia high school graduates earning each of the top five career and 
technical education (CTE) credentials or no CTE credential, by graduation year 
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MOS is Microsoft Office Specialist. 

Note: Figure presents the percentage of each year’s graduates (any diploma type) who earned each of the top five credentials or no credential. Percentages 
can total to more than 100 percent each year because students can earn multiple credentials. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


The top 10 credentials the 2017 graduates earned were mostly broad credentials that are not 
narrowly aligned to a specific occupation or industry. 

Eight of the top 10 credentials (for all graduates and for both types of diploma) were an industry credential, and 
most were broad credentials that are not narrowly aligned to a specific occupation or industry (table 3). Only 
two state licensures (National Nurse Aide Program and Cosmetology) were in the top 10 credentials (for all 
graduates or for either of the two diploma types), and the percentage of students earning these credentials was 
1.1 percent or less (see table 3). 


Table 3. Number and percentage of 2017 graduates earning the top 10 career and technical education 
credentials, for all graduates and by diploma type 


Noh 7-Taveccrosiaule|(=cmel[o)o)aar-) 
All graduates Standard diploma graduates graduates 
(oig-tol-Tardt-] Marlentem WAle alr) 
Department of Education Rank Percent Number Rank Percent Number Rank Percent Number 
credential type) 
Broad credentials 


W!se Financial Literacy 
Certification (Industry) 


Workplace Readiness Skills 
(WRS) 


MOS PowerPoint (Industry) 
MOS Word (Industry) 
College and Work Readiness 
Assessment (Industry) 
National Career Readiness 
Certificate (Industry) 
MOS Excel (Industry) 


Narrowly aligned credentials 


Customer Service and Sales 
Certification (Industry) 
Armed Services Vocational 


Aptitude Battery (Industry) : 13 1,93 6 eee ag oS 
NenepsiNdiee Seer eeem 10 1.0 851 na na na 10 1.0 498 
(State licensure) 
Cosmetology Licensure (State ce as a 9 An 375 re - ae 
licensure) 
ServSafe Manager Certification a fe fia 10 Ag 361 a “a 
(Industry) 

Number of students na na 88,519 na na 35,604 na na 48,743 


na is not applicable and is displayed for credentials not in the top 10. MOS is Microsoft Office Specialist. 

Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Percentages within a column can total to more than 100 percent because students can earn multiple credentials. Shading reflects the percentage of students 
earning the credential. The darkest shade indicates percentages greater than 10, the middle shade indicates percentages from 2 to 10, and the lightest shade 
indicates percentages less than 2. All graduates includes all students who graduated in 2017, regardless of diploma type. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


For 2017 graduates, there were few differences between the top 10 CTE credentials by diploma type. 


Although the CTE credential is required only for the Standard diploma, there were few differences across diploma 
types in the percentage of students earning credentials or in the types of credentials students most commonly 
earned. Eight CTE credentials were in the top 10 for both Standard and Advanced Studies diploma graduates, and 
the percentages by diploma type were mostly within 3 percentage points of each other (see table 3). The only 
percentage-point differences greater than 3 were for the WRS and W!se credentials. Standard diploma graduates 
earned the WRS credential more often (40.6 versus 36.2 percent), while Advanced Studies diploma graduates 
earned the W!se certification more often (70.4 versus 63.4 percent). 


The Cosmetology Licensure and ServSafe Manager Certification were in the top 10 for Standard diploma graduates 
but not Advanced Studies diploma graduates, though only a small percentage of Standard diploma graduates 
earned these credentials (1.1 and 1.0 percent, respectively). The MOS Excel and National Nurse Aide Program 
credentials were in the top 10 for Advanced Studies diploma graduates but not Standard diploma graduates; only 
2.6 and 1.0 percent of Advanced Studies diploma graduates (respectively) earned these credentials. 
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English learner students and students with disabilities earned the top 10 credentials at lower rates 
than other Standard diploma graduates in 2017. 


The largest differences across the subgroups were in the rates at which English learner students and students with 
disabilities earned the WRS and W!se credentials. While 63 percent of Standard diploma graduates earned the 
W!se Financial Literacy Certification, only 55 percent of English learner students and 56 percent of students with 
disabilities earned this credential. Also, approximately 33 percent of English learner students earned the WRS, 
compared to 41 percent of all graduates and 39 and 38 percent of economically disadvantaged students and 
students with disabilities. The other subgroup percentages were mostly within 3 percentage points of each other 
and within 3 percentage points of all Standard diploma graduates (table 4). There were several differences across 
the groups in terms of which credentials appeared in the top 10; however, these differences were all for 
credentials earned by fewer than 2.0 percent of Standard diploma graduates (see table 4). 


Table 4. Percentage of 2017 Standard diploma graduates earning the top 10 career and technical education 
credentials, by federal program participation. 


PNT eTater-Tae) English Economically 


(ole) Coyaare) learner disadvantaged Students with 
Credential name (Virginia Department of Education credential type) graduates students students disabilities 
Broad credentials 


W!se Financial Literacy Certification (Industry) 
Workplace Readiness Skills (WRS) 

MOS PowerPoint (Industry) 

National Career Readiness Certificate (Industry) 
MOS Word (Industry) 

College and Work Readiness Assessment (Industry) 1.0 na na na 
MOS Excel (Industry) na 0.9 0.9 0.7 


Narrowly aligned credentials 
(eee) 


Customer Service and Sales Certification Assessment (Industry) 


Armed Services Vocational Aptitude Battery (Industry) 1.6 na 1.5 0.9 
Cosmetology Licensure (State licensure) 1.1 0.4 na na 
ServSafe Manager Certification (Industry) 1.0 0.4 72 1.0 
National Nurse Aide Assessment Program (State licensure) na na 1.0 na 
ASE Student Certification Automotive: Maintenance and Light Repair na 0.6 na 0.8 
(Industry) 

Number of students 35,604 3,636 21,686 8,099 


na is not applicable and is displayed for credentials that are not in the top 10. MOS is Microsoft Office Specialist. ASE is Automotive Service Excellence. 
Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Percentages within a column can total to more than 100 percent because students can earn multiple credentials. Shading reflects the percentage of students 
earning the credential. The darkest shade indicates percentages greater than 10, the middle shade indicates percentages between 2 and 10, and the lightest 
shade indicates percentages less than 2. Refer to figure C1 in appendix C for a visual presentation of the top five credentials by federal program participation 
and table C1 in appendix C for the top 10 credentials by federal program participation. Federal program participation refers to participation in federal 
programs for English learner students, economically disadvantaged students, and students with disabilities, and includes students who were eligible for 
these programs at any point during their enrollment in a Virginia high school. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Black and Hispanic students were less likely than White and Asian students to earn the broad WRS 
and W!se Financial Literacy credentials. 

The largest differences across race/ethnicity were in the rates at which students earned the WRS and Wlse 
credentials. White and Asian graduates earned the Wlse certification at higher rates than Black and Hispanic 
graduates (67.1 and 66.2 percent versus 58.7 and 59.9 percent, respectively). White graduates earned the WRS 
credential more often than Asian and Black graduates (42.8 percent versus 37.2 and 37.3 percent, respectively). 
The difference between the rate at which Black and Hispanic graduates earned the Customer Service and Sales 
Certification Assessment is also greater than 3 percentage points (5.6 and 1.9 percent, respectively). Finally, the 
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difference between the percentages of Black and Asian graduates earning the National Career Readiness 
Certificate was also greater than 3 points (5.2 and 1.0 percent, respectively). The remaining differences were all 
less than 3 percentage points. 


Table 5. Percentage of 2017 Standard diploma graduates earning the top 10 career and technical education 
credentials, by race/ethnicity 


PNTES Taver-lae| 
(ole) toyaare) 
Credential name (Virginia Department of Education credential type) graduates White INSET A) Black? Hispanic® 
Broad credentials 


W!se Financial Literacy Certification (Industry) 
Workplace Readiness Skills (WRS) 

MOS PowerPoint (Industry) 

MOS Word (Industry) 

National Career Readiness Certificate (Industry) 


College and Work Readiness Assessment (Industry) 1.0 na 1.0 na 0.8 
MOS Excel (Industry) na na 1.8 na na 
SkillsUSA: Customer Service Examination (Industry) na na na 1.3 na 
AAFCS: Broad Field Family and Consumer Sciences (Industry) na na na 0.8 na 


Narrowly aligned credentials 


Customer Service and Sales Certification Assessment (Industry) a ae ee 1.9 


Armed Services Vocational Aptitude Battery (Industry) 1.6 1.9 0.6 1.4 1.0 
Cosmetology Licensure Examination (State licensure) 1.1 1.5 na na 1.0 
ServSafe Manager Certification (Industry) 1.0 na na 1.3 na 
National Nurse Aide Assessment Program (State licensure) na 1.6 na na na 
ASE Student Certification Automotive: Maintenance and Light Repair na 1.3 0.6 na nz, 
(Industry) 

Number of students 35,604 16,531 1,197 11,004 5,153 


na is not applicable and is displayed for credentials that are not in the top 10. MOS is Microsoft Office Specialist. ASE is Automotive Service Excellence. AAFCS 
is American Association of Family and Consumer Sciences. 

a. Black is defined as a person having origins in any of the Black racial groups of Africa. 

b. Hispanic is defined as a person of Cuban, Mexican, Puerto Rican, South American, Central American, or other Spanish culture or origin, regardless of race. 
Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Percentages within a column can total to more than 100 percent because students can earn multiple credentials. Shading reflects the percentage of students 
earning the credential. The darkest shade indicates percentages greater than 10, the middle shade indicates percentages between 2 and 10, and the lightest 
shade indicates percentages less than 2. Refer to figure C2 in appendix C for a visual presentation of the top five credentials by race/ethnicity and table C2 
in appendix C for the top 10 credentials by race/ethnicity. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Numerous differences existed in the top 10 credentials for 2017 Standard diploma graduates by 
region, especially in the Southside region where the National Career Readiness Certificate was more 
common than the WRS and W!se Financial Literacy credentials. 


For credentials that were in the top 10 in every region of Virginia, the percentage of students earning these 
credentials varied widely, most notably in region 8 (Southside) (table 6). The percentage of students earning the 
Wlse Financial Literacy Certification credential ranged from 70.2 percent in region 1 (Central Virginia) to 37.8 in 
region 8 (Southside). The W!se credential was the most commonly earned credential in every region except region 
8 (Southside), where the National Career Readiness Certificate had the highest percentage. The WRS credential 
ranged from 50.3 percent in region 3 (Northern Neck) to 9.0 percent in region 8 (Southside). The WRS credential 
was the second most commonly earned credential in every region except region 8 (Southside), where it was 
fourth. The MOS PowerPoint credential ranged from 2.3 percent in region 1 (Central Virginia) to 12.7 percent in 
region 8 (Southside). Finally, the MOS Word credential ranged from 1.9 in region 1 (Central Virginia) to 7.7 percent 
in region 7 (Southwest). 
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Table 6. Percentage of 2017 Standard diploma graduates earning the top 10 career and technical education 
credentials, by region 
(oig=tol-Tard-] marclentem WAles lal) Central Northern Northern Western 
Department of Education Virginia Tidewater INTeXel.g Virginia Valley Virginia Southwest Southside 
credential type) (Region1) (Region 2) (Region3) (Region 4) (Region5) (Region6) (Region7) (Region 8) 


Broad credentials 
W!se Financial Literacy 
Certification (Industry) 
Workplace Readiness Skills 
(WRS) 
MOS PowerPoint (Industry) 
MOS Word (Industry) 
National Career Readiness 
Certificate (Industry) 
College and Work Readiness 
Assessment (Industry) 
MOS Excel (Industry) 
SkillsUSA: Customer Service 
Examination (Industry) 
AAFCS: Broad Field Family and 
Consumer Sciences (Industry) 


na na na na na na na 


na 1.6 na na na na na na 


Narrowly aligned credentials 


Customer Service and Sales 
Certification (Industry) 
Armed Services Vocational 
Aptitude Battery (Industry) 
Cosmetology Licensure (State 
licensure) 
ASE Student Certification 

Automotive: Maintenance and 1.1 na 1.4 0.6 1.5 na na na 
Light Repair (Industry) 

National Nurse Aide Assessment 


na 


1.3 1.9 


na na 1.6 0.7 1.5 na na 


Program (State licensure) is a oe ie 
Beef Quality Assurance 
Certification (Industry) a di es ne 
NCCT Core: Introductory Craft 
Skills (Industry) oe sis bi 
Private Applicator Certification 
: na na na na 
(State licensure) 
ServSafe Manager Certification 17 0.9 tia 4a 
(Industry) 
ASE Student Certification 
Automotive: Suspension and na na 1.6 na na na na na 
Steering (Industry) 
ASE Student Certification 
Automotive: Brakes (Industry) ie i ae oa oe me di ies 
Cosmetology Assessment 
(NOCTI) na na na na na na na 
Number of students 5,361 7,573 2,437 11,111 3,042 2,500 2,458 1,001 


na is not applicable and is displayed for credentials that are not in the top 10. MOS is Microsoft Office Specialist. ASE is Automotive Service Excellence. AAFCS 
is American Association of Family and Consumer Sciences. NCCT is National Construction Career Tests. NOCTI is National Occupational Competency Testing 
Institute. 

Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Regions are Virginia’s superintendent regions. Percentages within a column can total to more than 100 percent because students can earn multiple 
credentials. Shading reflects the percentage of students earning the credential. The darkest shade indicates percentages greater than 10, the middle shade 
indicates percentages between 2 and 10, and the lightest shade indicates percentages less than 2. See figure C3 in appendix C for a visual presentation of 
the top five credentials by region and table C3 in appendix C for the top 10 credentials by region. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 
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There were also numerous differences in 
terms of which credentials were in the top 10 


Narrowly aligned credentials appearing in the top 10 in one or more 
regions (regions shown in parentheses): 


by region, and more narrowly aligned 
credentials emerged when looking at the top 
10 credentials across regions (see table 6). 
Region 3 (Northern Neck) and Region 7 
(Southwest) had more narrowly aligned 
credentials in their top 10 than any other 
region. Region 7 (Southwest) had the largest 
percentages for narrowly aligned credentials 
in the top 10 credentials; however, the 
percentages are still low (3.6 percent or less) 
compared to the percentages for the W!se 
and WRS assessments. Region 2 (Tidewater), 
on the other hand, had the smallest number 
of narrowly aligned credentials in the top 10. 


Armed Services Vocational Aptitude Battery (Central Virginia, Tidewater, 
Northern Neck, Northern Virginia, Western Virginia, and Southside) 

ASE Student Certification Automotive: Brakes (Northern Neck and Valley) 
ASE Student Certification Automotive: Suspension and Steering 
(Northern Neck) 

Beef Quality Assurance Certification (Southwest) 

Cosmetology Assessment (Southwest) 

Cosmetology Licensure (Northern Neck, Northern Virginia, Valley, and 
Southwest) 

Customer Service and Sales Certification ((Central Virginia, Tidewater, 
Northern Neck, Northern Virginia, Western Virginia, and Southwest) 
National Nurse Aide Assessment Program (Valley and Southwest) 

NCCT Core: Introductory Craft Skills (Valley and Southwest) 

Private Applicator Certification (Western Virginia and Southside) 


ServSafe Manager Certification (Central Virginia, Tidewater, Western 


Virginia, and Southside) 


Limitations 


This study is descriptive in nature and cannot establish the reasons for the increase in the percentage of students 
earning CTE credentials over the seven-year span included in the study. The percentage of graduates earning CTE 
credentials increased in the years before and after policy implementation. The increases may have been, in part, 
a response to other factors, such as the regular legislative updates related to credentials which increased the 
perceived importance of credentials for graduates’ long-term success, changes in the labor market, a shift in 
students’ or school staff’s understanding of postsecondary readiness, or the addition of the Economics and 
Personal Finance course (which is tied to the W!se Financial Literacy Certification) to graduation requirements. 


In addition, this study may be underestimating the percentage of graduates who earned CTE credentials. Each 
year some Virginia students secure an exception to the Standard diploma CTE credential requirement by 
participating in competency-based instruction in lieu of an assessment. The data do not distinguish between 
graduates with exemptions and graduates who did not earn a CTE credential for other reasons. 


Implications 


To assess the impact of their CTE policies, Virginia’s CTE stakeholders and policymakers could benefit from 
collecting information that helps them understand the value of different credentials to ensure the state’s CTE 
policies are improving students’ career readiness. This study’s findings highlight the particularly urgent need to 
gather evidence to determine the value of the WRS and W!se Financial Literacy Certification credentials, as these 
were the most commonly earned credentials. While limited, the available empirical evidence suggests that general 
career readiness credentials may hold little value in the labor market and licenses are likely more valuable 
(ExcelinED & Burning Glass Technologies, 2019). For CTE credentials to meet their goal of increasing students’ 
workplace readiness or serving as a stepping-stone to more advanced certifications, they should align to labor- 
market needs and employers need to find them valuable in confirming that students are prepared for a career in 
the given field (ExcelinED & Burning Glass Technologies, 2019; Sublett & Griffith, 2019; Harris, Warner, Yee, & 
Wilkerson, 2020). This type of information can also help students better understand the value of particular 
credentials and make more informed decisions about which ones to pursue. 


There are several strategies that Virginia’s CTE stakeholders and policymakers can use to identify high-value 
credentials, and they can use this information to update CTE instructional programs and policies as needed. 
Suggested strategies include gathering data on labor-market demand for credentials, identifying credentials that 
confer credit toward postsecondary education and training, and building an agreed-upon list of credentials that 
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spans the education and workforce systems (Education Strategy Group, Advance CTE, and Council of Chief State 
School Officers, 2018). Virginia stakeholders and policymakers could also consider working with local 
communities, employers, and postsecondary institutions to identify valuable credentials. Finally, they could 
potentially examine employment and postsecondary outcomes across schools where students earn different 
types of credentials to gain a better understanding of the value of various credentials. 


Virginia’s stakeholders and policymakers also need to collect information about the value of credentials to help 
them determine the implications of their CTE policies on educational equity. Because Black, Hispanic, and high- 
needs students, such as English learner students, economically disadvantaged students, and students with 
disabilities, are more likely to earn a Standard diploma, the CTE credential requirement can potentially improve 
their lifelong employment outcomes. Whether the credentials benefit such students depends on the value of the 
credentials. This study found that a smaller percentage of Black, Hispanic, English learner students, and students 
with disabilities earned the W!se Financial Literacy Certification. Thus, determining the value of the W!se 
credential is particularly important. 


Virginia stakeholders and policymakers can use the regional findings from this study and information about the 
value of credentials to inform their decisionmaking. Across regions, the number of narrowly aligned credentials 
appearing in the top 10 ranged from three (in the Tidewater region) to six (in the Northern Neck and Southwest 
regions). As there is some evidence that narrowly aligned credentials are more valuable than broad credentials 
(ExcelinED & Burning Glass Technologies, 2019), Virginia could examine the regions in which more students earn 
narrowly aligned credentials to see if they are doing something different to encourage students to earn these 
credentials. Further, given the large variety in the top 10 credentials by region, Virginia stakeholders could benefit 
from analyzing the extent to which the top credentials in each region align with local labor-market needs (see for 
example, Harris et al., 2020). 


The findings of this study highlight the importance of examining the types of credentials students earn and the 
need to determine their value. After the passage of the Every Student Succeeds Act, many more states moved to 
include industry-recognized credentials as part of their accountability system (Education Strategy Group, Advance 
CTE, and Council of Chief State School Officers, 2018). This shift in accountability is only as valuable as the 
credentials students earn. 
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Appendix A. About the study 


This section provides additional information about the study. It begins with a description of the relevance of career 
and technical education (CTE) credentials and goes on to describe the Virginia context, including information 
about the state’s policies relevant to this study. 


CTE credential relevance 

Attention to high school credentialing in state CTE policies has expanded rapidly in recent years (Education 
Strategy Group, Advance CTE, and Council of Chief State School Officers, 2018). In 2002, approximately 10 states 
had policies on industry-recognized credentials in secondary education (Castellano, Stone, & Stringfield, 2005, 
citing Workforce Excellence Network, 2002). Between 2013 and 2015, 36 states adopted policies regarding such 
credentials (Association for Career and Technical Education [ACTE] & National Association of State Directors of 
Career Technical Education Consortium [NASDCTEC], 2014, 2015, 2016). 


Florida’s 2007 landmark Career and Professional Education Act reflected recognition of the importance of 
providing high school students with opportunities to earn CTE credentials to smooth transitions to college and 
careers (Goodman, Meyer, & Imperatore, 2014). By 2013, 42 states had K-12 pathways leading to CTE credentials 
(National Center for Education Statistics, 2016), and in 2016, 11 states included attainment of industry-recognized 
credentials in their school accountability systems (Advance CTE, 2016). 


States offering CTE credentials have reason to hypothesize that they will boost postsecondary enrollment and 
completion as well as improve workforce outcomes. Assessments that lead to industry credentials have the 
potential to signal employers that individuals are prepared for entry-level employment (Bartlett, 2004). Because 
CTE credentialing assessments require students to demonstrate industry-recognized levels of occupational skills 
and competencies, many states also see potential for such assessments to add rigor to high school CTE programs 
and improve high school graduates’ readiness for postsecondary education and training. 
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The Virginia context 

Previous research shows that Virginia’s Standard diploma earners are less likely to enroll in, persist in, or complete 
postsecondary education programs, including one-, two-, and four-year programs, compared with students who 
earned a college preparatory diploma (the Advanced Studies diploma). Research has shown this to be true for 
Standard diploma graduates who completed CTE programs of study (Jonas, Garland, & Yamaguchi, 2014; 
Yamaguchi, Garland, & Jonas, 2014) and Standard diploma graduates overall (Garland, LaTurner, Herrera, Ware, 
Jonas, & Dougherty, 2011; Holian & Mokher, 2011; Jonas & Garland, 2014). For example, researchers estimated 
that within four years of high school graduation, fewer than 8 percent of Virginia’s Standard diploma earners in 
the graduating class of 2008 had earned any type of college credential, compared with 46 percent of Advanced 
Studies diploma earners (Jonas & Garland, 2014). 


In an economy where most jobs require some type of postsecondary education or training (Carnevale, Smith, & 
Strohl, 2013), the relatively low postsecondary enrollment and completion rates of Standard diploma earners raise 
concerns about the value of the Standard diploma. In response, Virginia joined states across the country looking 
to strengthen the college and career readiness of their high school graduates by adding CTE credentialing 
assessments to the requirements for earning a Standard diploma. 


For several years, Virginia’s career and technical education programs have offered students credentialing 
assessments (CTE credentials) and opportunities to earn state licensure in some programs. The state has 
conducted a long-term effort to increase the availability of these assessments in high school (figure A1). For 
example, legislation enacted in 2009 required the Virginia Board of Education to identify industry credentialing 
assessments that could substitute for assessments that were already required for graduation (Code of Virginia, 
2009). Starting with the 2009/10 school year, Virginia began reporting the number of credentialing assessments 
that students took and passed in the annual Career and Technical Education Statewide Annual Performance 
Report, Perkins IV Performance Standards (Virginia Department of Education [VDOE], 2011). In 2011, the 
legislature required the Virginia Board of Education to develop a plan for increasing the number of students 
receiving industry credentials and state licensures as part of career and technical education programs, and in early 
2011 Virginia passed a law requiring students who entered grade 9 for the first time in 2013 or later and who earn 
Virginia’s Standard diploma to earn a CTE credential by passing a Virginia Board of Education—approved 
assessment to graduate (Code of Virginia, 2011).° 


5 The Standard diploma is one of two federally recognized diplomas in Virginia. The other is the Advanced Studies diploma. See the Key 
Terms box (box 1) in the body of the report for additional information on these two diplomas. 
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Figure A1. Timeline of major Virginia legislative, state board, and department of education activities related to 
career and technical education (CTE) credentials, 2009-2017 


Virginia legislature Legislature requires the Legislature requires 2016 legislation requires all 
requires the state Virginia Board of students who entered students starting grade 9 for 
Board of Education Education to develop a grade 9 for the first time the first time in 2013 and 
to identify industry plan to increase the in 2013 or later to earna graduating in 2018/19 to 
credentialing rape nOnericents CTE credential by passing either (i) complete an 
assessments receiving industry a Virginia Board of Advanced Placement, honors, 
students can use Predenialancisiare Education—approved or International Baccalaureate 
toward high school licensure as part of CTE assessment to graduate course, or (ii) earn a CTE 
graduation programs (Code of with Virginia’s Standard credential. The new 
requirements (Code Virginia, 2011). diploma (Virginia requirement applies to all 
of Virginia, 2009). Administrative Code, n.d.). diplomas (VDOE, n.d.b). 
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Virginia Department of Workplace Readiness Skills . The first class required to pass The first 
Education (VDOE) begins earcerentis ane ‘ The first grade i d | cohort of on- 
pted for statewide eileeecennited an Economics and Persona : 
reporting the number of CTE use (Career and Technical Education Finance course to earn a time 
credentials students took and Consortium of States, 2011). noe ‘A “Ie diploma (Standard or graduates 
passed in the annual Career See Advanced Studies) enters high required to 
and Technical Education The W!se Financial Literacy graduate with a school. Local school divisions earn a CTE 
Statewide Annual certification is approved for use as a : Standard may use the W!se Financial credential to 
Performance Report, Perkins Board of Education—approved aun enters Literacy Certification as part earna 
IV Performance Standards credential (VDOE, 2011). high school. of the course. Standard 
(VDOE, 2011). diploma. 


VDOE is the Virginia Department of Education. 
Note: The 2016 legislation does not apply to the students included in this study. 


Source: Authors’ compilations from various sources cited above. 


The rationale for this policy was in part to raise the bar on CTE knowledge and skills learned in high school and, by 
so doing, to strengthen Standard diploma earners’ preparation for careers and/or postsecondary education and 
training. In addition to viewing the credentialing requirement as a way to increase students’ attainment in 
workplace and technical skills (VDOE, 2016), the credentials can serve as a stepping-stone toward achieving more 
advanced certification that may require additional postsecondary education and training (VDOE, 2008). Adding 
the CTE credential as a graduation requirement within the Standard diploma was a substantial change in policy 
that impacted the graduating class of 2017 and beyond, potentially requiring 40 percent of high school graduates 
(approximately 34,000 students annually) to earn a CTE credential (VDOE, n.d.). 


Students can meet the requirements by passing one of four types of Board of Education—approved assessments. 
They can earn an industry-recognized credential issued by a third party such as a business (for example, Cisco, 
Microsoft), trade association (for example, National Healthcareer Association), or industry group (for example, 
National Institute for Automotive Service Excellence). These parties provide an independent assessment of 
specific technical competencies and knowledge needed for a specific industry or job. Students can also satisfy the 
requirement by passing a state licensure examination required for certain occupations (for example, Cosmetology 
Licensure Examination administered by the Virginia Board of Barbers and Cosmetology). Students can also take a 
National Occupational Competency Testing Institute assessment that measures the specific knowledge and skills 
needed for a particular occupation (for example, Accounting [Basic or Advanced] assessment). Finally, students 
can satisfy the requirement by passing the Virginia Workplace Readiness Skills assessment, which measures more 
general skills needed for employment. 
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Appendix B. Methods 


This section describes in more detail the methods used by this study. 


Research questions 


This study addressed the following research questions: 


1. For the 2011-2017 graduates, what were the top five career and technical education (CTE) credentials 
earned and how did they change over time? 

2. For the 2017 graduates, how many of each of the top 10 CTE credentials did students earn overall and by 
diploma type? 

3. For 2017 Standard diploma earners, how did the top 10 CTE credentials vary across different student 
characteristics (such as demographics and federal program participation) and by region? 


Data sources 


The study used several student-level data elements from the Virginia Longitudinal Data System (VLDS) (table B1). 
VLDS is a federated data system that permits authorized users to merge de-identified state administrative data 
from multiple Virginia state agencies, including data from the Virginia Department of Education (VDOE). 
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Table B1. Variables in the analysis 
Career and technical education (CTE) data 
CTE exam identifier A number that identifies the CTE credential exam. CTE credential exam 
identifiers were matched to credential names that the Virginia Department of 
Education provided. 


CTE program of study An indicator for whether the student completed a CTE program of study. 

completer 

Result of exam An indicator for whether the student passed the CTE credential exam. Students 
earned the credential if they passed the exam. 

CTE credential type Categorical variable including the following: Industry, National Occupational 


Competency Testing Institute, state licensure; and Workplace Readiness Skills. 
Student demographics 

Race/ethnicity Categorical variable including the following: White, American Indian, Asian, Black, 
Hispanic, Hawaiian/Pacific Islander, and multiracial. The analysis excludes American 
Indian, Hawaiian/Pacific Islander, and multiracial because of the small number of 
students in these categories. If a student had conflicting race records, the analysis 
used the most common race. Hispanic or Latino is defined as a person of Cuban, 
Mexican, Puerto Rican, South American, Central American, or other Spanish culture 
or origin, regardless of race. Black or African American is defined as a person having 
origins in any of the Black racial groups of Africa. 


Federal program participation 


English learner status An indicator for whether the student was identified as an English learner at any 
point in a Virginia high school. 

Economically disadvantaged An indicator for whether the student was identified as economically 

status disadvantaged at any point in a Virginia high school. 

Disability status An indicator for whether the student was identified as a student with a 


disability at any point in a Virginia high school. 

Graduation data 

Diploma type Categorical variable including the following: Standard diploma, Standard 

Technical diploma, Advanced Technical diploma, Advanced Studies diploma, 
Special diploma, International Baccalaureate diploma, and Modified Standard 
diploma. If a student had records for multiple diploma types, Advanced diploma 
records were selected over Standard diploma records, and Standard diploma 
records were selected over the remaining types of diploma. 


Graduation year Year in which the student earned a diploma. If a student had graduation records 
from multiple years, the analysis used grade 12 records over other grades. 
District and school ID ID for the responsible district and school that the student attended upon 


graduation. District ID was used to identify the region the student was in. 


Note: All data came from the Virginia Longitudinal Data System (2010/11-2016/17). 
Source: Authors’ analysis and Virginia Longitudinal Data System Data Dictionary (2019). 


The state regularly conducts automated validity checks throughout data collection, a process that minimizes or 
eliminates missing data on student characteristics. For example, the data verification process requires records to 
include information about students’ race/ethnicity, gender, and other demographic variables. Furthermore, all 
students who take state assessments have data included in the state data system. Therefore, if a student record 
lacks assessment data, either the student did not take the test or there is a data error, but this does not necessarily 
mean that data are missing. For example, students who do not take the Algebra Il assessment can still earn a 
Standard diploma. A student’s record therefore can be complete even if there is no score on the Algebra Il 
assessment. 


VDOE collects, analyzes, and reports these data as part of state and federal accountability requirements, and the 
data are subject to audit. This data reporting includes credentialing data, which Virginia includes in accountability 


reporting for CTE programs’ federal accountability.© This combination of factors is typically associated with 
increased data quality (Jonas, Goldschmidt, & Garland, 2014) and is likely to reduce error. 


Sample and selection criteria 


The study population and selection criteria varied by research question. Research question 1 included all students 
who graduated from high school at some point between 2011 and 2017. VDOE did not start collecting student- 
level CTE credential data until the 2007/08 school year, when it began collecting a snapshot of these data at the 
end of each year. Thus, the 2011 graduating class is the first group of students for which the CTE credential data 
are complete. The study runs through the 2017 cohort, as the graduates of 2017 were the first to complete high 
school with the new requirement in place, and this was the last year that had complete data at the time of the 
analyses. Although the policy first applied to 2017 Standard diploma graduates, the analyses included earlier years 
and other diploma types to capture overarching trends in CTE credentials. 


The study team narrowed the sample for research questions 2 and 3 to produce results that can be more easily 
interpreted while maintaining a focus on the first graduates subject to the new CTE credential graduation 
requirement. The sample for research questions 2 and 3 included only 2017 high school graduates. The sample 
for research question 3 is further narrowed to include only 2017 Standard diploma graduates. Appendix C includes 
the results for Advanced Studies diploma graduates to provide stakeholders with additional information that they 
may find valuable. 


Analysis plan 

This study used a descriptive approach to address the research questions, as described in this section. Using 
straightforward numbers provides stakeholders with the status of the population in terms of demographic 
characteristics and high school outcomes and also helps frame the results in the context of the student population 
at each time point. Furthermore, because the sample included the universe of students in the target population 
during the period being studied, one can reasonably take a population perspective on these data rather than treat 
them as a sample.’ 


Research question 1 


To answer research question 1, the study team calculated the percentage of graduates, by year, who earned each 
CTE credential, as shown in equation B1. 


N earnedcy 
cy — 
Ny 


P1 (B1) 


P1.,is the percentage of students who graduated in year y (y = 2011, ..., 2017) who earned CTE credential c. After 
calculating the percentage of students who earned each credential in each year, the study team identified the five 
credentials with the largest percentage in each year. The analysis included a comparison of the top five credentials 
and the percentage of students earning them from 2011 to 2017. 


Research question 2 


To answer research question 2, the study team calculated the percentage of 2017 graduates, by diploma type, 
who earned each CTE credential, as shown in equation B2. 


6 Virginia’s Career and Technical Education Annual Performance Reports are available at 
http://www.doe.virginia.gov/instruction/career_technical/statistics reports/index.shtml. These reports contain outcomes 
by school division and regional Career and Technical Education Centers. Virginia uses five federally approved technical- 
skills-attainment performance measures; industry credentials are a component of four of these measures. 

7 Seastrom (2017) describes potential benefits and drawbacks of treating state administrative data as a population or a 
sample from which one can derive estimates for purposes of school accountability. Similar principles apply here. 
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a ea nee (B2) 


P2-a is the percentage of students who graduated in 2017 who earned diploma type d (d = any diploma, 
Standard diploma, Advanced Studies diploma) and earned CTE credential c. After calculating the percentage of 
students who earned each credential by diploma type, the study team identified the 10 credentials with the 
largest percentage for each diploma type. The analysis included a comparison of the top 10 credentials and the 
percentage of students earning them by diploma type. This part of the analysis includes the top 10 credentials to 
provide stakeholders with additional information about which credentials students earned once a credential was 
required for the Standard diploma. 


Research question 3 
To answer research question 3, the study team calculated the percentage of 2017 Standard diploma graduates 


who earned each CTE credential, by student characteristic, as shown in equation B3. 


P3¢5 = eo (B3) 


P3,, is the percentage of 2017 Standard diploma graduates who earned CTE credential c and have the student- 
level characteristic s. After calculating the percentage of students who earned each credential by student 
characteristic, the study team identified the 10 credentials with the largest percentage for each student 
characteristic. The analysis included a comparison of the top 10 credentials and the percentage of students 
earning them by student characteristic. The analysis included the following student-level characteristics: 


e Student demographics 
o Race/ethnicity 
e Federal program participation 
o English learner status 
© Economically disadvantaged status 
o Identified as a student with disabilities receiving services in high school (Individualized Education Program) 


The study team also calculated the percentage of high school graduates who earned each CTE credential, by 
region, as shown in equation B4. 


P3 cr, = meaner (B4) 


P3-, is the percentage of 2017 Standard diploma graduates who earned CTE credential c from region r (r = 1, ..., 
8). After calculating the percentage of students who earned each credential by region, the study team identified 
the 10 credentials with the largest percentage for each region. The analysis included a comparison of the top 10 
credentials and the percentage of students earning them by region. 
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Appendix C. Supporting analyses 

This appendix includes analyses that support the findings in the report. This appendix also extends the analysis for 
research question 3 to 2017 Advanced Studies diploma graduates (tables C1—C2). Stakeholders may wish to 
compare these results to those of Standard diploma graduates (tables 4—6 in the report), or they may find value 
in better understanding the types of credentials earned by Advanced Studies diploma graduates. Students can 
earn more than one credential, which is why the percentages in tables C1 and C2 can total to more than 100 
percent for a particular group of students. 


Table C1. Percentage of 2017 Advanced Studies diploma graduates earning the top 10 career and technical 
education credentials, by federal program participation 
INT Ne WeTareccte| English Economically 


Credential name (Virginia Department of Education Studies diploma learner disadvantaged Students with 
credential type) graduates students students disabilities 
Broad credentials 


W!se Financial Literacy Certification (Industry) 
Workplace Readiness Skills (WRS) 

MOS Word (Industry) 

MOS PowerPoint (Industry) 

College and Work Readiness Assessment (Industry) 
MOS Excel (Industry) 

National Career Readiness Certificate (Industry) 


International Baccalaureate for Business & 
Management (Industry) 
Narrowly aligned credentials 


Customer Service and Sales Certification (Industry) 


Armed Services Vocational Aptitude Battery (Industry) sly, 1.6 i) 1.1 
National Nurse Aide Assessment Program (State 1.0 na 1.7 na 
licensure) os 
Certification of Pharmacy Technicians (State licensure) na 1.2 na na 
Advanced Placement Computer Science (Industry) na 1.1 na 1.8 
Number of students 48,743 1,316 13,089 2,678 


na is not applicable and is displayed for credentials that are not in the top 10. MOS is Microsoft Office Specialist. 

Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Percentages within a column can total to more than 100 percent because students can earn multiple credentials. Shading reflects the percentage of students 
earning the credential. The darkest shade indicates percentages greater than 10, the middle shade indicates percentages between 2 and 10, and the lightest 
shade indicates percentages less than 2. Federal program participation refers to participation in federal programs for English learner students, economically 
disadvantaged students, and students with disabilities, and includes students who were eligible for these programs at any point during their enrollment in 
a Virginia high school. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Table C2. Percentage of 2017 Advanced Studies diploma graduates earning the top 10 career and technical 
education credentials, by race/ethnicity 


PN WNe WZ-Talercte! 
Credential name (Virginia Department of Studies diploma 
sXe [ULer-yu(olamerg=Xe(=lald i] may] oY=)) graduates Hispanic? 
Broad credentials 
W!se Financial Literacy Certification (Industry) 
Workplace Readiness Skills 
MOS Word (Industry) 
MOS PowerPoint (Industry) 
College and Work Readiness Assessment 
(Industry) 
MOS Excel (Industry) 
National Career Readiness Certificate (Industry) 
International Baccalaureate for Business & 
Management (Industry) 
Narrowly aligned credentials 


Customer Service and Sales Certification 
(Industry) 


Armed Services Vocational Aptitude Battery 1.2 na na 1.8 15 
(Industry) 

National Nurse Aide Assessment Program (State 1.0 1.2 na na na 
licensure) ee 
Autocad Certified User: Autodesk (Industry) na 1.0 1.0 na na 
Advanced Placement Computer Science na na na na 
(Industry) 

ServSafe Manager Certification (Industry) na na na 1.2 na 
Introduction to Engineering Design: Project Lead na na na na 1.1 


the Way (Industry) 
Number of students 48,743 29,387 4,369 7,841 4,658 


? Black or African American is defined as a person having origins in any of the Black racial groups of Africa. 

® Hispanic or Latino is defined as a person of Cuban, Mexican, Puerto Rican, South American, Central American, or other Spanish culture or origin, regardless 
of race. 

na is not applicable and is displayed for credentials that are not in the top 10. MOS is Microsoft Office Specialist. 

Note: The study team considered a credential to be narrowly aligned if it supports preparation for a specific occupation or industry and broad otherwise. 
Percentages within a column can total to more than 100 percent because students can earn multiple credentials. Shading reflects the percentage of students 
earning the credential. The darkest shade indicates percentages greater than 10, the middle shade indicates percentages between 2 and 10, and the lightest 
shade indicates percentages less than 2. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Appendix D. Other analyses 


This appendix presents other analyses not directly tied to the research questions but that provide additional 
information to stakeholders that they will likely find valuable. While the research questions focused on the top 
credentials earned, other credential-earning patterns might emerge across student groups. These other analyses 
looked for differences across student groups in the types of credentials earned (tables D1 and D2), the number of 
credentials earned (figures D1 and D2), and the percentage of students completing a CTE program of study (figures 
D3 and D4). These analyses examined 2017 graduates by diploma type. Students can earn more than one 
credential, which is why the percentages in tables D1 and D2 can total to more than 100 percent for a particular 
subgroup. Of the students who earned a Standard diploma in 2017, 26 percent earned two credentials and 13 
percent earned more than two credentials (figures D1 and D2 in appendix D). 


Table D1. Percentage of students earning each career and technical education credential type in 2017, by 


diploma type and federal program participation 
Syr-Tatet-laemel|el(elaar- Advanced Studies diploma 


Credential type All English Economically Students All English Economically Students 
graduates learner disadvantaged with graduates learner disadvantaged with 
students students disabilities students students disabilities 

W!se Financial Literacy 63 55 60 56 70 68 73 71 

Certification? 

Workplace Readiness 41 33 39 38 36 40 39 36 

Skills 

Industry 26 15 26 22 27 19 30 24 

National Occupational 4 1 4 3 4 2 5 4 

Competency Testing 

Institute ——— 

State licensure 3 1 3 2 2 2 3 1 
Number of students 35,604 3,636 8,099 21,686 48,743 1,316 2,678 13,089 


® The Virginia Department of Education technically classifies the W!se Financial Literacy Certification as an industry credential; however, the table shows the 
percentages for this specific credential separately from other industry credentials to highlight the fact that it accounts for the largest proportion of all 
credentials. 

Note: Percentages within a column can total to more than 100 percent because students can earn multiple credentials. All graduates includes all students 
who graduated in 2017 with their respective diploma type. Federal program participation refers to participation in federal programs for English learner 
students, economically disadvantaged students, and students with disabilities, and includes students who were eligible for these programs at any point 
during their enrollment in a Virginia high school. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 
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Table D2. Percentage of students earning each career and technical education credential type in 2017, by 
diploma type and race/ethnicity 


Sielalet-lacmell ed(eyaare NeW eTalesxeMsiablel (mello leyaare 
Credential type rN} INSETS} Black Hispanic All White NSETo) Black Hispanic 
graduates graduates 

WISE Financial Literacy 63 67 66 59 60 70 70 61 74 72 

Certification? 

Workplace Readiness 41 43 37 37 41 36 35 33 37 43 

Skills 3 
cIndustry HD 

National Occupational 4 6 2 3 1 4 5 2 4 3 

Competency Testing 

Institute 

State licensure 3 5 1 2 2 2 2 1 3 2 

Number of students 35,604 16,531 1,197 11,004 5,153 48,743 29,387 4,369 7,841 4,658 


° The Virginia Department of Education technically classifies the W!se Financial Literacy Certification as an industry credential; however, the table shows the 
percentages for this specific credential separately from other industry credentials to highlight the fact that it accounts for the largest proportion of all 
credentials. 

Note: Percentages within a column can total to more than 100 percent because students can earn multiple credentials. All graduates includes all students 
who graduated in 2017 with their respective diploma type. Hispanic or Latino is defined as a person of Cuban, Mexican, Puerto Rican, South American, 
Central American, or other Spanish culture or origin, regardless of race. Black or African American is defined as a person having origins in any of the Black 
racial groups of Africa. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Figure D1. Percentage of graduates who earned one, two, three, four or more, or no career and technical 
education credential in 2017, by diploma type and federal program participation 
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Note: All graduates includes all students who graduated in 2017 with their respective diploma type. Federal program participation refers to participation in 
federal programs for English learner students, economically disadvantaged students, and students with disabilities, and includes students who were eligible 
for these programs at any point during their enrollment in a Virginia high school. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Figure D2. Percentage of graduates who earned one, two, three, four or more, or no career and technical 
education credential in 2017, by diploma type and race/ethnicity 


100 
12 12 2 

90 

80 

70 

60 27 

Y 
o 

Qo 50 
o) 
a 

40 

30 

ui) 

20 

10 

0 

All White Asian Black Hispanic All White Asian Black Hispanic 
graduates graduates 
Standard Diploma Advanced Studies Diploma 
Number 35,604 16,531 1,197 11,004 5,153 48,743 29,387 4,369 7,841 4,658 
of students: 


@1 @2 B83 O4ormore ONone 


Note: All graduates includes all students who graduated in 2017 with their respective diploma type. Hispanic or Latino is defined as a person of Cuban, 
Mexican, Puerto Rican, South American, Central American, or other Spanish culture or origin, regardless of race. Black or African American is defined as a 
person having origins in any of the Black racial groups of Africa. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 
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Figure D3. Percentage of 2017 Standard and Advanced Studies diploma graduates who completed a career 
and technical education program of study, by federal program participation 
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Note: All graduates includes all students who graduated in 2017 with their respective diploma type. Federal program participation refers to participation in 
federal programs for English learner students, economically disadvantaged students, and students with disabilities, and includes students who were eligible 
for these programs at any point during their enrollment in a Virginia high school. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


Figure D4. Percentage of 2017 Standard and Advanced Studies diploma graduates who completed a career 
and technical education program of study, by race/ethnicity 
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Standard Diploma Advanced Studies Diploma 


Number 35,604 16,531 1,197 11,004 5,153 48,743 29,387 4,369 7,841 4,658 
of students: 


Note: All graduates includes all students who graduated in 2017 with their respective diploma type. Hispanic or Latino is defined as a person of Cuban, 
Mexican, Puerto Rican, South American, Central American, or other Spanish culture or origin, regardless of race. Black or African American is defined as a 
person having origins in any of the Black racial groups of Africa. 

Source: Authors’ calculations using data from the Virginia Longitudinal Data System. 


